Combining reinforcement learning and optimal control for the control of nonlinear dynamical systems

This thesis presents a novel hierarchical learning framework, Reinforcement Learning Optimal Control, for controlling nonlinear dynamical systems with continuous states and actions. The adapted approach mimics the neural computations that allow our brain to bridge across the divide between symbolic action-selection and low-level actuation control by operating at two levels of abstraction. First, current findings demonstrate that at the level of limb coordination human behaviour is explained by linear optimal feedback control theory, where cost functions match energy and timing constraints of tasks. Second, humans learn cognitive tasks involving learning symbolic level action selection, in terms of both model-free and model-based reinforcement learning algorithms. We postulate that the ease with which humans learn complex nonlinear tasks arises from combining these two levels of abstraction. The Reinforcement Learning Optimal Control framework learns the local task dynamics from naive experience using an expectation maximization algorithm for estimation of linear dynamical systems and forms locally optimal Linear Quadratic Regulators, producing continuous low-level control. A high-level reinforcement learning agent uses these available controllers as actions and learns how to combine them in state space, while maximizing a long term reward. The optimal control costs form training signals for high-level symbolic learner. The algorithm demonstrates that a small number of locally optimal linear controllers can be combined in a smart way to solve global nonlinear control problems and forms a proof-of-principle to how the brain may bridge the divide between low-level continuous control and high-level symbolic action selection. It competes in terms of computational cost and solution quality with state-of-the-art control, which is illustrated with solutions to benchmark problems.Open Acces

DialettiBot: a Telegram Bot for Crowdsourcing Recordings of Italian Dialects

In this paper we describe DialettiBot, a Telegram based chatbot for crowdsourcing geo-referenced voice recordings of Italian dialects. The system enables people to listen to previously recorded audio and encourages them to contribute to building a collective linguistic resource by sending voice recordings of their own spoken dialects. The project aims at collecting a large sample of voice recordings in order to promote knowledge of linguistic variation and preserve proverbs or idioms typical for different local dialects. Moreover, the collected data can contribute to several voice-based Natural Language Processing (NLP) applications in helping them understand utterances in non-standard Italian.In questo articolo descriviamo DialettiBot, un chatbot basato su Telegram per raccogliere registrazioni audio georeferenziate di dialetti italiani. Il sistema permette alle persone di ascoltare le registrazioni precedentemente inserite, e le incoraggia a contribuire alla costruzione di questa risorsa linguistica collettiva, attraverso l’invio di registrazioni audio nel proprio dialetto. Il progetto mira a raccogliere una grande mole di registrazioni che possono aiutare a promuovere la conoscenza delle variazioni linguistiche e la salvaguardia dei proverbi o modi di dire tipici di ogni dialetto locale. I dati raccolti possono inoltre contribuire a diverse applicazioni del trattamento automatico del linguaggio (TAL) che hanno bisogno di essere adattate per comprendere espressioni dialettali

Genetic Classification of ore-forming processes

The absence of a genetic classification of ore-forming processes is the result of a rather slow-paced accumulation of knowledge in ore-forming processes, which, in its turn, could be explained by objective conditions such as the fact that during the last few decades the genetic classification included only the description of mineral deposits. The existing genetic mineral deposit classifications has developed and embraced the genetic classification of ore-forming processes in accordance with system theory, structured taxons revealing the nature of the processes and the basis- information source of these processes. The base involved the developed model system of ore mineralization- ore formations (geological formations with syngenetic mineralization) within poly-component and mono-component subformations; for convergent mineralization- geological types of deposits. The latter, as well as non-convergent subformations has accumulated all data about initiating and conditioning ore formation within wide-scaled geological processes. The geological mineralization types within ore formations and subformations are included in the genetic classification of ore-forming processes. This, in its turn, makes it possible to forecast the functions and provide a regular transition into the geological-genetic classification of ore-forming processes in accordance to above-mentioned matrix-structure, and, simultaneously, further the development of the existing geological- genetic theory of ore formation

Seasonal progression of active-layer thickness dependent on microrelief

Introduction Active-layer thickness is a major factor for all physical and biological processes in permafrost soils. It is closely related to the fluxes of energy, water and carbon between permafrost landscapes and the atmosphere. Active-layer thickness is mainly driven by air temperature, but also influenced by snow cover, summer rainfall, soil properties and vegetation characteristics (Nelson et al., 1998). The typical polygonal tundra of the Lena Delta is characterised by a pronounced microrelief, which causes a high small-scale heterogeneity of soil and vegetation properties. Consequently, also the active-layer thickness varies substantially across small lateral distances of decimetres to metres. In order to up-scale results of process studies to the landscape scale, a quantification of the heterogeneity of active-layer thickness is of great interest

Automatic Labeling of Phonesthemic Senses

Abstract This study attempts to advance corpus-based exploration of sound iconicity, i.e. the existence of a non-arbitrary relationship between forms and meanings in language. We examine a number of phonesthemes, phonetic groupings proposed to be meaningful in the literature, with the aim of developing ways to validate their existence and their semantic content. Our first experiment is a replication o